Creating Reproducible Documents with Quarto

About Data Services

We help faculty, researchers, and students find, use, manage, visualize and share data.

https://dataservices.library.jhu.edu/

dataservices@jhu.edu

About you

Using the Zoom ✅ reaction,

Have you used Quarto before?
Have you used RMarkdown before?

About you

Please open the etherpad (link in the chat), and answer the following questions

Do you use a programming language (such as R, Python, etc.) for your research? If so, which one?
What is bringing you to this training? What kinds of documents or outputs are you hoping to use Quarto to create?

Today

Background
Quarto tour
Hands-on portion: Creating documents & presentations
Demo portion: Creating manuscripts & beyond
Resources & further learning

Artwork from “Hello, Quarto” keynote by Julia Lowndes and Mine Çetinkaya-Rundel, presented at RStudio Conference 2022. Illustrated by Allison Horst.

Quarto overview

Quarto is an authoring framework for data science.
It is a tool that allows you to create a variety of output documents (HTML, PDF, MS Word, MS PowerPoint, and more!) from a combination of text and code (in R, Python, Julia, and more).

Like this

Here’s an example

Here’s even more examples

Quarto as next-generation R Markdown

Quarto unifies and extends the R Markdown ecosystem
- unifies for people who love R Markdown and all its spinoffs
- extends for people who don’t know R Markdown

Quarto: Why?

We can think of notebooks as documents that mix prose, code, and output.
Providing notebooks as curated research outputs would greatly enhance transparency and reproducibility.

Page from a laboratory notebook of Alexander Graham Bell, 1876.

The image on the slide is a notebook that belonged to Alexander Graham Bell, an inventor and scientist who is credited with patenting the first telephone.
The notebook contains a diagram of some kind of system, in addition to notes about how the system works. With the level of detail present, we can maybe imagine that someone could create this system themselves based on the diagram and Bell’s annotation of it.
So what does this have to do with Quarto? Quarto’s goal is to make it easy to create and distribute computational notebooks.
When I say notebooks, you might immediately think of Jupyter notebooks, which are definitely notebooks, but we’re thinking more broadly of notebooks as documents that combine prose, code, and outputs in order to make it easy to follow the creator’s thought process and ultimately, reproduce their results.
We can imagine that if notebooks were accepted as research outputs at the same level as peer-reviewed articles, that would increase the transparency and reproducibility of research.
Current peer-review and publication workflows do not readily support notebooks as research outputs, or encourage their use. This is for a number of different reasons, one of the most foundational being that there isn’t one tool or platform that makes it easy to create flexible, collaborative computational notebooks.
Quarto is posit’s effort to create free, open-source, easy-to-use infrastructure for reproducible research.

Quarto: How?

Quarto documents can be edited and rendered in RStudio, Jupyter Notebooks, and Visual Studio, and via the Quarto Command Line Interface.
What happens when a Quarto document is rendered:

Quarto engines 🚂

Quarto engines do two things:
1. Execute code
2. Render text, code, and output into markdown (“make it pretty”)
Quarto’s current engines (knitr and Jupyter) can execute R, Python, Julia, ObservableJS, and many more languages.
You can select an engine or R will automatically detect which engine to use based on your document. Functionally, the engines are extremely similar, and are almost identical in syntax

Quarto engines

knitr 🧶

Renders R and Julia natively.
Renders python via the reticulate package
Comes installed with RStudio.
Best for Quarto beginners or RMarkdown converts.

Jupyter 🪐

Renders R, Python, Julia, and many more languages natively.
Requires installation of Jupyter, in addition to any other kernels you might.
Best for those who are familiar with Jupyter.

Quarto tour

When you create a Quarto document, you will see options for many different type of output documents, and a place to designate the engine you would like to use.
- We recommend not setting an engine. This will allow Quarto to dynamically select an engine based on the content you have in your document.
The YAML header controls the document format and is automatically generated based on options you select when creating your document.
- There are thousands of YAML header options that vary by output type, a good reference here.

Open RStudio
Navigate to the directory you want to work in, make a directory called Quarto - show how to do this in Files
Create a new R Project in the directory
Select New File > Quarto Document
Fill out title and author
Note the engine selection and explain that they should leave this as-is because Quarto detects which engine to use based on the code chunks present.
At a base level, both the knitr and jupyter engines can execute all of the Markdown and R or Python code. For more obscure languages, support may differ by engine. There are also very slight differences in some of the chunk options you can specify that don’t become relevant until you are using Quarto in a very advanced way, and will never break your code.
Point out all of the possible output formats, within document and also on the sidebar. Note that LaTex is require for PDF outputs.
Create the document
Toggle between the source and visual and note that we will only be working in the visual editor. Note that if you are familiar with RMarkdown this is a huge improvement and creates a lower barrier to entry in a word processor-like environment
Go over the YAML header. YAML is a markup language that is interpreted by the Quarto engines. The YAML header controls the document format and there are thousands of options for how Quarto can display things, listed here https://quarto.org/docs/reference/, we will edit the YAML header later.
Go over the contents of the document and render it (make sure to go to Render Settings > Viewer Pane), giving it a title when prompted.

Text in documents

Create headings and text by typing in the visual editor
Insert citations by DOI, crossref search, or from your Zotero library with Insert > Citation or with @
Insert footnotes with Insert > Footnote
Add tabsets to separate components of your analysis with Insert > Tabset

Delete everything in the template document except the YAML header
Create the headings Introduction, Data Processing and Exploratory Data Analysis
Under Introduction, add “This is a notebook for the Scurvy Lab at JHU to share code and results. The Scurvy Lab studies the efficacy of different nutrients on treating scurvy.”
Under Data Processing, add “This is where we will process our data for analysis.”
Under Exploratory Data Analysis, add “This is where we will explore our data.”
Render - show in Files that what it created was an html document because we said output should be html.
Set toc: true in the YAML header (show auto-complete feature) and render, show that toc now shows up
Add citation in Introduction (search Treatise on Scurvy in citation manager and pick 1953). Note that you can also pull citations directly from Zotero if you have it running on your computer. Note that bibliography is added to the YAML header.
Add tabsets under Exploratory Data Analysis: Gum Rot, Skin Sores, Knee Weakness, Lassitude, render

Code in documents

Insert executable code with Insert > Executable Cell - you can combine programming languages in one document
Specify options for displaying code chunks with #| at the beginning of the code chunk (see next slide).
Specify options for displaying code chunks globally in the YAML header with execute: followed by the option.

Code execution options

when set to false (i.e. `eval: false`)	run code	show code	show output	show warning	show errors
`eval`	❌	✔️	❌	❌	❌
`echo`	✔️	❌	✔️	✔️	✔️
`output`	✔️	✔️	❌	❌	❌
`include`	✔️	❌	❌	❌	❌
`warning`	✔️	✔️	✔️	❌	✔️
`error`	✔️	✔️	✔️	✔️	❌

Add an R code chunks under Introduction to load libraries (note that we want this as high up in our document as possible to make our data available)

library(medicaldata) library(dplyr) library(ggplot2) library(reticulate) scurvy <- medicaldata::scurvy
Open up the scurvy dataset, describe what it is and note that we want to turn the severity values into numeric variables.
Add another R code chunk under Data Processing that turns the symptom rankings into numeric variables scurvy_processed <- scurvy |> mutate_at(c(4:8), ~as.numeric(substr(., start = 1, stop = 1)))
- Render it and note that we are seeing the code we wrote in addition to all of R’s messaging when we load our packages
- Just like the whole document has YAML that tells it how to display, each code chunk can also have YAML telling it how to display
- To suppress this we can add a code chunk option with #| output: false (this is code-chunk specific YAML)
- Show what happens if we do #| echo: false
- Show that we could do #| include: false if we don’t want to include the code or warnings
- We will settle on #| warning: false since that is exactly what we want to suppress.
- Decide that we don’t want to display any warnings in the entire document, so we will put the warning: false option into the YAML header under execute: .
- Decide that we only want to display the data processing code, not loading libraries and reading data. Add |# include: false to the libraries chunk.

Other code options

You can add annotations to your code by inserting a number tag, like # <1>, next to the line of code you want to annotate. Then, immediate under the code chunk, add the number followed by the annotation, like 1. [Insert annotation text]
To allow users to expand or hide code chunks, you can add the option code-fold: true under format: html: in the YAML header, or as an options for a specific code chunk with #| code-fold: true

Inline code ⛸️

You can add inline code to your document text with the syntax {r} 2 + 2 surrounded by backticks `
The above will render in your Markdown to 4

Using Python and R together 🐍

Quarto can execute Python and R together in the same document with the reticulate R package
When you add a Python code chunk to a Quarto document, R will ask if you want to create a reticulate environment for the document. This will manage the Python version and packages you have access to in the document.
Once you do this, you will see that you now have a Python Console and Environment in addition to R.
You can reference R objects in Python with r. , and vice versa with py$.

Add R data visualization code under Gum Rot to create a plot of gum rot severity on day 6 by treatment
- scurvy_processed |> group_by(treatment) |> summarise(gum_rot_severity = mean(gum_rot_d6)) |> ggplot() + geom_col(aes(treatment, gum_rot_severity))
Add Python data visualization code under Skin Sores to create a plot of skin sore severity on day 6 by treatment - note that you can reference objects in the R environment with r.
- scurvy_processed = r.scurvy_processed
- scurvy_processed.groupby('treatment').skin_sores_d6.mean().plot(kind = "bar")
Show that when I add a Python code chunk, Quarto will ask me if I want to create a reticulate environment in R. After this finishes running, show that I now have a Python console and a Python environment in my RStudio
List the packages I have with py_list_packages() (do this in the R console, not the Python one, because the function comes from the reticulate package, which is installed in R) - note that we will need pandas and matplotlib for our visualization, show how to install with py_install("packagename")

Referencing outputs in documents

You can make figures produced by code chunks can be referenced from elsewhere in your document by specifying the code chunk option #| label: and giving it a label with the prefix fig- .
The output can then be referenced in your document text with @fig-label .
Images, tables, and numbers can also be referenced, but the syntax differs - see here for a full reference.

Slides in Quarto

One of the nicest things about Quarto is that you can easily convert between output formats
- Go from document to slides by changing format: html to format: revealjs in the YAML header
- Slides are delineated by headings
Create Quarto slides from scratch with New File > Quarto Presentation

Quarto slides features

Add slide notes with Insert > Slide Notes
Speaker View allows for easy presenting, and PDF export mode allows for easy sharing
You can add the ability to annotate slides with format: revealjs: chalkboard: true

Quarto Dashboards

Like Shiny, but with a focus on layout and organization of components without native reactivity or interactivity.
Interactivity can be added to dashboards with Shiny components.
See this presentation on Quarto Dashboards for more!

Quarto manuscripts 📜

Let’s explore a Quarto manuscript
In RStudio:
- Create a manuscript with New Project > New Directory > Quarto Manuscript
- Add new journal format in the Terminal window with quarto add quarto-journals/plos, full list of journal formats here, and change outputs in _quarto.yml

Quarto is trying to make it easier to integrate notebooks into expected research outputs
Link manuscript example https://quarto-ext.github.io/manuscript-template-jupyter/ and explore
Show how you would create a manuscript with RStudio > New Project > New Directory > Quarto Manuscript (point out other formats as well), but then navigate to the La Palma example (switch to the manuscript project)
Go over the front matter in the qmd YAML header - note how much more information we have here
Go over the _quarto.yml, this controls some manuscript-specific settings for the document, like output formats, which we will look at later
Render the manuscript in the Viewer Pane - note the download options, and how much of the content is similar to what we just learned
for the agu template, we had to run quarto add template quarto-journals/agu . See all journal templates here - open link https://github.com/quarto-journals
Note that you can also supply options for what stage your manuscript is - preprint, review, doubleblind
Note that you can add multiple journal formats to a single Quarto document and it will convert it for you and pull the relevant front matter
Add another journal format to the manuscript - do quarto add template quarto-journals/elsevier - then navigate to quarto.yml and add elsevier-pdf: default
Re-render and show that the new format shows up as a download option.

Referencing external notebooks

Within your manuscript, create a folder called notebooks
Insert any .qmd or .ipynb notebooks that you’d like to reference in your manuscript
You can embed code and/or output in your manuscript without runnning it every time you render you manuscript, go to Insert > Shortcode and type embed notebooks/NotebookName.qmd#cell-label

Artwork from “Hello, Quarto” keynote by Julia Lowndes and Mine Çetinkaya-Rundel, presented at RStudio Conference 2022. Illustrated by Allison Horst.

Further learning

thank you! 🙏

Please let us know what you thought of this workshop at bit.ly/quarto-survey

Any questions?

Upcoming workshops:

Data Bytes: The Hidden Mapping Powers of ArcGIS Arcade: 11/18 12-1pm
Finding a Repository to Share Research Data: 12/3 12-1pm